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DETAILED ACTION 

Applicants request for reconsideration filed 01/12/07 has been entered 
and carefully considered. However, arguments regarding rejections under 
35.U.S.C 102 and 35. U.S.C 103 to claims (1-7, 9-15, 17-46, 48-54, 56-82, 84- 
90 and 92-1 1 1 ) have not been found to be persuasive. Therefore, these claims 
are rejected under the same ground of rejection as set forth in the Office Action 
mailed (08/09/06). 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United 
States only if the international application designated the United States and was published under 
Article 21 (2) of such treaty in the English language. 

Claims 1-2, 4-5, 13-15, 17, 37, 40-41, 43-44, 52-54, 56, 76-77, 79-80, 
88-90 and 92 are rejected under 35 U.S.C. 102(e) as being anticipated by Jain 
et al. [US. 6,567,980]. 

As to claims 1 , 37, 40 and 76, Jain et al. disclose a computer implemented 
method and corresponding apparatus for displaying multimedia information 
comprising the steps/means for displaying a graphical user interface (GUI) on 
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the display (column 4, lines 20-25); displaying, in a first area of the GUI, a 
representation of the multimedia information stored by the multimedia document 
(figure 2, 172), the displayed representation of the multimedia information 
comprising a representation of information of the first type and a representation 
of information of the second type (figure 2, the frame 172 contains both the first 
type: image and the second type: caption text); displaying a first lens covering a 
first portion of the first area (figure 2, the first lens covers the frame 172); 
displaying, in a second area of the GUI (figure 2, area 176), a representation of 
multimedia information displayed in the first portion of the first area (figure 2, 
one of the frame of area 176), the representation of multimedia information 
displayed in the second area comprising a portion of the representation of 
information of the first type covered by the first lens and a portion of the 
representation of information of the second type covered by the first lens (figure 
2, in the capture process of the video in area 172, the frame 172 is captured 
and extracted from the process, then it is displayed in the second display 176. 
So, one of the multiple keyframes in the second display 176 is the extracted 
keyframe of the first display 172, see column 3, line 50 through column 4, line 
40). 

As to claims 2, 41 and 77, Jain et al. also disclose displaying a first thumbnail 
image in the first area of the GUI (figure 2, 1 72), the first thumbnail image 
comprising the representation of information of the first type; and displaying a 
second thumbnail image in the first area of the GUI, the second thumbnail 
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image comprising the representation of information of the second type (figure 2, 
column 4, lines 20-65). 

* 

As to claims 4, 43 and 79, Jain et al. show determining a first time and a 
second time associating with the first lens; displaying, in the second area of the 
GUI, a representation of information of the first type occurring between the first 
time and the second time associated with the first lens; and displaying, in the 
second area of the GUI, a representation of information of the second type 
occurring between the first time and the second time associated with the first 
lens (figure 2, 178, column 6, line 40-60 and column 7, lines 15 through column 
8, line 60). 

As to claims 5, 44 and 80, Jain et al. also show receiving user input moving 
the first lens to cover a second portion of the first area; and responsive to the 
user input, automatically changing the information displayed in the second area 
of the GUI such that the representation of multimedia information displayed in 
the second area of the GUI corresponds to the representation of multimedia 
information included in the second portion of the first area (figure 2, receiving 
the user input to select another keyframe of the video in the first display 1 72 
and then display in the second display 176). 

As to claims 13, 52 and 88, Jain et al. also teach the information of the first 
type corresponding to video information (column 2, lines 6-24); and the 
representation of the information of the first type comprising one or more video 
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keyframes extracted from the video information (column 2, lines 17-22 and 
column 4, lines 20-40). 

As to claims 14, 53 and 89, Jain et al. provide the information of the second 
type corresponding to audio information (column 2, lines 18-20); and the 
representation of information of the second type comprises text information 
obtained from transcribing the audio information (column 2, lines 15-25). 

As to claims 15, 54 and 90, Jain et al. also provide the information of the 
second type corresponding to closed-caption (CC) text information; and the 
representation of information of the second type comprises text (column 4, lines 
20-50). 

As to claims 17, 56 and 92, Jain et al. also provide receiving input indicating 
selection of a portion of the multimedia information occurring between a first 
time and a second time; and performing a first operation on the portion of the 
multimedia information occurring between a first time and a second time 
(column 6, lines 40-60 and column 12, lines 30-50). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented 
and the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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Claims 3, 6-7, 9-12, 18-36, 38-39, 42, 45-46, 48-51, 57-75, 78, 81-82, 
84-87 and 93-1 1 1 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Jain et al. [US. 6,567,980]. 

As to claims 3, 42 and 78, Jain et al. disclose displaying the portion of the 
representation of information of the first type covered by the first lens in the 
second area of the GUI; and displaying the portion of the representation of 
information of the second type covered by the first lens in the second area of 

T 

the GUI (figure 2, 172 (first area)). Jain et al. fail to clearly teach a first panel 

f. 

and a second panel. However, it was well known in the art that the first and 
second panels are displayed in the GUI area (see figure 2, first area 172, the 
first portion contains image, the second portion contains text). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to 
display the first and second information types in the first and second panels in 
the second GUI area in order to provide a good and convenient GUI. 

As to claims 6-7, 45-46 and 81-82, Jain et al. fail to clearly teach the third area 
of the GUI. However, it was well known in the art to display the third area in the 
GUI. It would have been obvious to one of ordinary skill in the art at the time of 
the invention to display the third area of the GUI in order to provide a good and 
convenient GUI. 

As to claims 9, 22, 48, 61, 84 and 97, Jain et al. disclose receiving user input 
moving the first lens to cover a first portion of the first area and responsive the 
user input, automatically changing the information displayed in the second area 
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of the GUI (figure 2, the user can selects another keyframe in the first display 
172 and then display it in the second display 176). But, Jain et al. fail to clearly 
teach a third area in order to automatically change the information displayed in 
the third area of the GUI. However, it was well known in the art to display the 
third area in the GUI. It would have been obvious to one of ordinary skill in the 
art at the time of the invention to display the third area of the GUI in order to 
provide a good and convenient GUI. 

As to claims 10, 49 and 85, Jain et al. disclose receiving a user input moving 
the first lens to cover a second portion of the first area and responsive to the 
user input, automatically changing the information displayed in the second area 
of the GUI corresponds to the representation of multimedia information included 
in the second portion of the first area (figure 2, the first lens cover first and 
second portions of the keyframe in the first display 172). Jain et al. fail to clearly 
teach the information displaying in the third area of the GUI. However, it was 
well known in the art to display the third area in the GUI. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to display 
the third area of the GUI in order to provide a good and convenient GUI. 

As to claim 11, Jain et al. teach displaying a sub-lens covering a portion of the 
first area of the GUI corresponding to the first portion of the second area of the 
GUI covered by the second lens (figure 2, column 6, 42-51 , the sub-lens is used 
to select any other keyframe in the first display 172). 
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As to claim 12, Jain et al. also teach receiving a user input moving the second 
lens to cover a second portion of the second area; and responsive to the user 
input, automatically changing a position of the sub-lens to cover a portion of the 
first area of the GUI corresponding to the second portion of the second area 
(figure 2, column 6, 42-51 , the sub-lens is used to select any other keyframe in 
the first display 172). 

As to claims 18, 38, 57 and 93, Jain et al. disclose displaying a graphical user 
interface, displaying, in a first area of the GUI, a representation of the 
multimedia information stored by the multimedia document occurring between a 
start time and an end time associated with the multimedia document (figure 2, 
178 and time line 180). Each of the video images occurs between a start time to 
an end time (column 6, lines 1-25). But, Jain et al. fail to clearly teach the step 
of displaying a first lens emphasized a portion of the first area of the GUI, the 
portion of the first area emphasized by the first lens comprising a representation 
of multimedia information occurring between a first time (t1 ) and a second time 
(t2). 

However, it was well known in the art to display a first lens emphasizing a 
portion of the first area of the GUI, the portion of the first area emphasizing by 
the first lens comprising a representation of multimedia information occurring 
between a first time (t1 ) and a second time (t2). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to have the portion of the first area emphasized by the first lens 
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. comprising a representation of multimedia information occurring between a first 
time (t1) and a second time (t2) in order to provide a good and convenient GUI. 

As to claims 19, 58 and 94, In light of rejection of the third area, Jain et al. 
show displaying a second lens emphasizing a portion of the second area of the 
GUI, the portion of the second area emphasized by the second lens comprising 
a representation of multimedia information occurring between a third time (t3) 
and a fourth time (t4) (each video image has different start-end time. So, a first 
image has its start-end time (t1-t2) and a second image has its start-end time 
(1344). 

Jain et al. fail to clearly teach the step of displaying, in a third area of the GUI, 
the representation of multimedia information occurring between t3 and t4, the 
representation of multimedia information displayed in the third area comprising 
a representation of information of the first type occurring between t3 and t4 and 
a representation of information of the second type occurring between t3 and t4. 

However, it was well known in the art to display the third area in the GUI. It 
would have been obvious to one of ordinary skill in the art at the time of the 
invention to display the third area of the GUI in order to provide a good and 
convenient GUI. 

As to claims 20, 59 and 95, Jain et al. fail to clearly teach a fifth and sixth time, 
however, in light of rejection of t1 , t2, t3 and t4, it was well known in the 
computer art to represent of multimedia information occurring between a t5 and 
t6. It would have been obvious to one of ordinary skill in the art at the time of 



Application/Control Number: 1 0/081 , 1 29 Page 
Art Unit: 2179 

the invention to display the third area of the GUI in order to provide a good and 
convenient GUI. 

As to claims 21, 23, 60, 62, 96 and 98, Jain et al. fail to clearly teach the step 
of changing the position of the second lens in response to user input such that 
the second lens emphasizes a portion of the second area of the GUI comprising 
a representation of multimedia information occurring between a fifth time (t5) 
and a sixth time (t6). However, in light of rejection of the third area and t5 and 
t6, it would have been obvious to a person of ordinary skill in the art at the time 
of the invention to change the position of the second lens in response to user 
input such that the second lens emphasizes a portion of the second area of the 
GUI comprising a representation of multimedia information occurring between a 
fifth time (t5) and a sixth time (t6). The motivation would have been to provide a 
convenient and good interface. 

As to claims 24, 63 and 99, Jain et al. show the information of the first type 
being video information; the information of the second type being audio 
information; the representation of the information of the first: type comprising 
one or more video keyframes extracted from the video information; and the 
representation of information of the second type comprising text information 
obtained from transcribing the audio information (column 2, lines 6-24, column 
2, lines 17-22 and column 4, lines 20-40 and column 2, lines 15-25). 

As to claims 25, 64 and 100, Jain et al. also show the information of the first 
type being video information; the information of the second type being 
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closed-caption (CC) text information; the representation of the information of the 
first type comprising one or more video keyframes extracted from the video 
information; and the representation of the information of the second type 
comprising text information included in the CC text information (column 4, lines 
20-50). 

As to claims 26, 65 and 101, Jain et al. disclose receiving information 
indicating a first topic; and analyzing the multimedia information stored in the 
multimedia document to identify one or more locations in the multimedia 
information that are relevant to the first topic (column 9, line 52 through column 
10, line 21); wherein displaying the representation of the multimedia information 
stored by the multimedia document occurring between ts and te in the first area 
of the GUI comprises highlighting the one or more locations in the multimedia 
information displayed in the first area of the GUI (column 8, lines 23-65); and 
wherein displaying the representation of multimedia information occurring 
between t1 and t2 in the second area of the GUI comprises highlighting the one 
or more locations in the multimedia information that occur between times t1 and 
t2 (column 6, lines 28-67). 

As to claims 27, 66 and 102, Jain et al. teach receiving input indicating 
selection of a portion of the multimedia information occurring between a 
selection start time and a selection end time; and performing a first operation on 
the portion of the multimedia information occurring between the selection start 
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time and the selection end time (column 6, lines 40-60 and column 12, lines 30- 
50). 

As to claims 28, 39, 67 and 103, Jain et al. disclose displaying a graphical 
user interface, displaying, in a first area of the GUI, a representation of the 
multimedia information stored by the multimedia document occurring between a 
start time and an end time associated with the multimedia document (figure 2, 
178 and time line 180). Each of the video images occurs between a start time to 
an end time (column 6, lines 1-25). Jain et al. also show displaying, in first 
section of a first area of the GUI, a first set of one or more video keyframes 
extracted from the video information occurring between a start time and an end 
time associated with the multimedia document (column 6, lines 42-60); in light 
of rejection of t1 and t2, it would have been obvious to a person of ordinary skill 
in the art at the time of the invention to display a first lens emphasizing a portion 
of the first section of the first area occurring between a first time and a second 
time and a portion of the second section of the first area occurring between t1 
and t2, the emphasized portion of the first section of the first area comprising a 
second set of one or more video keyframes extracted from the video 
information. The motivation would have been to provide a convenient and good 
interface. 

As to claims 29, 68 and 104, in light of rejection above about the displaying a 
second lens emphasizing a portion of the first section of the second area and a 
portion of the second section of the second area; and the video keyframes, it 
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was well known in the computer art to have the step of "the emphasized portion 
of the first section of the second area comprising a third set of one or more 
video keyframes extracted from the video information occurring between a third 
time (t3) and a fourth time (t4) and the third set of the video keyframes being a 
subset of the second set of one or more video keyframes. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to display 
the emphasized portion of the first section of the second area comprising a third 
set of one or more video keyframes extracted from the video information 
occurring between a third time (t3) and a fourth time (t4) in order to provide a 
good and convenient GUI. 

As to claims 30, 69 and 105, in light of rejection of claim 29, Jain et al. also 
teach outputting video information starting from t3 or from t4 or from a time 
between t3 and t4 in a first section of a third area of the GUI (column 6, lines 1- 
55 and column 13, lines 35-67); displaying text information corresponding to the 
information of the first type occurring between t3 and t4 in a second section of 
the third area of the GUI (column 14, lines 26-67). It would have been obvious 
to one of ordinary skill in the art at the time of the invention to display the 
emphasized portion of the first section of the second area comprising a third set 
of one or more video keyframes extracted from the video information occurring 
between a third time (t3) and a fourth time (t4) in order to provide a good and 
convenient GUI. 
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As to claims 31, 70 and 106, Jain et al. show the information of the first type 
being audio information (column 2, lines 10-25); and the text information 
corresponding to the information of the first type being obtained from 
transcribing the audio information (column 4, lines 40-67). 

As to claims 32, 71 and 107, Jain et al also show the information of the first 
type being closed-caption text information, and the text information 
corresponding to the information of the first type being extracted from the CC 
text information (column 4, lines 20-60 and column 8, line 50 through column 9, 
line 20). 

As to claims 33, 72 and 108, Jain et al. fail to clearly teach a first set of one or 
more slides extracting from the slides information occurring between ts and te. 
However, Jain et al. show the images instead. It was well known in the 
multimedia information that images can be displayed instead. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to 
display the images of the GUI in order to provide a good and convenient GUI. 

As to claims 34, 73 and 109, the claim is analyzed previously discuss with 
respect claims 29 and 33. p 

As to claims 35, 74 and 110, in light of rejection above, Jain et al. fail to clearly 
teach the step of " a first set of one or more whiteboard images image extract 
from the whiteboard images information occurring between ts and te". However, 
the whiteboard images were well known in the multimedia information 
environment. It would have been obvious to one of ordinary skill in the art at the 
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time of the invention to display the whiteboard images of the GUI in order to 
provide a good and convenient GUI. 

As to claims 36, 75 and 111, the claim is analyzed previously discuss with 
respect claims 28 and 35. 

As to claim 50, Jain et al. teach displaying a sub-lens covering a portion of the 
first area of the GUI corresponding to the first portion of the second area of the 
GUI covered by the second lens (figure 2, column 6, 42-51 , the sub-lens is used 
to select any other keyframe in the first display 172). 

As to claim 51, Jain et al. also teach receiving a user input moving the second 
lens to cover a second portion of the second area; and responsive to the user 
input, automatically changing a position of the sub-lens to cover a portion of the 
first area of the GUI corresponding to the second portion of the second area 
(figure 2, column 6, 42-51 , the sub-lens is used to select any other keyframe in 
the first display 172). 

As to claim 86, Jain et al. teach displaying a sub-lens covering a portion of the 
first area of the GUI corresponding to the first portion of the second area of the 
GUI covered by the second lens (figure 2, column 6, 42-51, the sub-lens is used 
to select any other keyframe in the first display 172). 

As to claim 87, Jain et al. also teach receiving a user input moving the second 
lens to cover a second portion of the second area; and responsive to the user 
input, automatically changing a position of the sub-lens to cover a portion of the 
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first area of the GUI corresponding to the second portion of the second area 
(figure 2, column 6, 42-51 , the sub-lens is used to select any other keyframe in 
the first display 172). 

Allowable Subject Matter 

Claims 8, 16, 47, 55, 83 and 91 are objected to as being dependent upon 
a rejected base claim, but would be allowable if rewritten in independent form 
including all of the limitations of the base claim and any intervening claims. 

Response to Arguments 

Applicant has argued the following points: 

1 ) Jain fails to teach displaying a first lens covering a portion of the first area 

2) Jain fails to teach a second area that- presenting portions of the first and 
second types of information covered by the first lens. 

The Examiner disagrees for the following reasons: 

1) Jain discloses a panel 172 displaying the live video being digitized, with play, 
stop... The frame portions are displayed in section 184 such as frames 186, 
188, 190, 192, 194 that are video clips of the live video. A user captures a 
desired portion (frame portion 1 72) of the live video including the frame portions 
of video clips. So, the live video includes video clips such as portions 172, 186, 
188, 190, 192, 194. The user puts a first lens covering the desired frame 172 by 
capturing and selecting the desired video clip frame 172. 
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If the user wants to capture a next video clip (like frame 186), the user puts a 
second lens on the frame 186 by selecting and then displaying this frame in the 
panel 172. Therefore, Jain discloses displaying the first lens covering a portion 
of the first area. 

2) Jain discloses multiple frame portioifs in the second area 176. Each portion 
includes two types of information: image and text. Text is displayed at the 
bottom of each image in the second area 176. Applicant's attention is directed 
to image 202. The text information is displayed at the bottom of the image 202. 

It is clearly that Jain discloses a second area that presenting portions of the first 
and second types of information covered by the first lens. 

Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension 
of time policy as set forth in 37 CFR 1 .1 36(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply 
is filed within TWO MONTHS of the mailing date of this final action and the 
advisory action is not mailed until after the end of the THREE-MONTH 
shortened statutory period, then the shortened statutory period will expire on 
the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .1 36(a) will be calculated from the mailing date of the advisory action. In 
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no event, however, will the statutory period for reply expire later than SIX 
MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications 
from the examiner should be directed to Mylinh Tran. The examiner can 
normally be reached on Mon - Thu from 7:00AM to 3:00PM at 571-272-4141. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Weilun Lo, can be reached at 571-272-4847. 

The fax phone numbers for the organization where this application or 
proceeding is assigned are as follows: 

571-273-8300 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see 
http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 
(toll-free). 
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